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ABSTRACT 


Passive learning environment to teach programming concepts, 
especially in large lecture classes hinders students’ motivation, 
performance and may adversely affect their achievement goals. 
The study presents the use of two instructional strategies 
teamwork and reflective thinking using educational technologies 
introduced in a class of 120 first year engineering students. This 
study also presents the impact of educational technologies based 
learning environment on students’ motivation, achievement 
goals, and engagement. 
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1. PROGRAM CONTEXT AND STATUS 


Iama third-year doctoral student at the School of Engineering 
Education at Purdue University with a research focus in both 
engineering and computer science education. For my dissertation 
research, I am conducting a quantitative investigation to study the 


Permission to make digital or hard copies of part or all of this work for personal or 
classroom use is granted without fee provided that copies are not made or distributed for 
profit or commercial advantage and that copies bear this notice and the full citation on 
the first page. Copyrights for third-party components of this work must be honored. For 
all other uses, contact the Owner/Author. 


ICER '19, August 12-14, 2019, Toronto, ON, Canada 
© 2019 Copyright is held by the owner/author(s). 
ACM ISBN 978-1-4503-6185-9/19/08. 
https://doi.org/10.1145/3291279.3339441 


321 


impact of educational technologies based learning environment 
on an introductory programming course student. I have collected 
the data for my dissertation. I plan to defend my dissertation in 
August 2020. I plan to submit papers based on my dissertation to 
renowned computer science education platforms. I would like to 
use the expertise of researchers at ICER’2019 DC to refine my data 
analysis strategies and to discuss possible future directions. As a 
member of the iOS development team of CourseMIRROR 
application, and I envision getting feedback on improving 
CourseMIRROR application. 


2. BACKGROUND AND MOTIVATION 


Actively involving students in the learning process is crucial 
for improving students’ understanding of fundamental STEM 
concepts and computer science education. However, most 
introductory engineering and computer science classes at the 
undergraduate level are taught in large lectures that typically 
result in traditional lecture-based instruction, which hinders 
students’ learning by placing them in mostly passive roles. 
Literature suggests the use of various educational technologies 
and instructional strategies to increase the active involvement and 
enrollment of students. In this study, I propose to combine these 
technologies and two of these instructional strategies together, 
which could enhance students’ learning outcomes in large 
engineering and computer science classes. The First-Year 
Engineering Program at large Midwestern University is designed 
with a team-based learning component. I for my study believes 
that we can combine this team based learning component with 
students’ reflective thinking, where students are prompted to 
reflect on their learning experiences. These two approaches 
individually have been effective to enhance students’ learning 
outcomes, but limited research exists that investigates the impact 
these two approaches together, specifically with the design and 
implementation of educational technologies component. 


3. RESEARCH QUESTIONS 


In this study, by using educational learning technologies, I 
wish to explore the relationships between students’ academic 
strategies in a First-Year 
engineering course. The main topics taught in the course are the 
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application of computer programming by using MATLAB, and the 
development of mathematical models to solve engineering 
problems in a collaborative teamwork manner. In addition, I am 
proposing to investigate how students’ achievement goals, 
engagement, and motivation change because of these experiences. 
Specifically, I am planning to address the following research 
questions: 


1. How do students’ team membership behaviors and reflection 
quality change over time? 

2. How do students’ reflection quality and team membership 
behaviors relate to their academic performances? 

3. How do students’ achievement goals, motivation, and 
engagement change because of these learning experiences? 

4. How do students’ achievement goals, motivation, and 
engagement relate to their academic performances? 


4. EDUCATIONAL TECHNOLOGIES 


I used two educational applications 1) CourseMIRROR (Mobile 
In-situ Reflections and Review with Optimized Rubrics) mobile 
learning system, and 2) CATME (Comprehensive Assessment of 
Team Member Effectiveness). I used these educational 
technologies for a variety of reasons such as to create an engaging 
learning environment for students, promoting students thinking 
about each delivered lecture and write the reflections, evaluate 
their own and peers team effectiveness after each team based 
project’s milestone, and also to collect data effectively. 


CourseMIRROR was used to collect students’ reflections after 
each lecture during one academic semester. CourseMIRROR 
prompts and scaffolds students to write insightful reflections on 
complex concepts and problems at the end of each lecture [1], [2]. 
It uses Natural Language Processing (NLP) algorithms to generate 
coherent summaries of reflections for each lecture by clustering 
them based on the common themes. Available to both instructors 
and students, these summaries allow instructors to identify and 
attend to the difficulties and misunderstandings that students 
encountered from the lecture. Further, students can see the larger 
picture of difficulties encountered by their peers. 


I also used CATME to form and manage student teams [3], [4]. 
The embedded peer evaluation feature of CATME allows team 
members to rate each teammate by using 5-level behaviorally 
anchored rating scales as well as self-evaluation of their 
performance. The peer evaluation system also provides feedback 
to students and to the instructor about how each team member is 
performing. 


5. DATA COLLECTION 


I have collected the data from 120 First-year engineering 
students. The dataset included: 3430 student reflections in 26 
lectures; team membership evaluation collected after each 
milestone (four times); pre and post surveys of students’ 
achievement goals, motivational constructs (self-efficacy, and task 
value), and engagement scales (behavioral, emotional, social, and 
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cognitive aspects); students’ background information (race and 
gender); and students’ individual exam scores in three exams. 


6. PROPOSED ANALYSIS 


I will conduct repeated measures ANOVA on reflection quality 
score and on team membership data to determine the changes 
over time. I will also use stepwise hierarchical regression analysis 
to explore the relationship between reflection quality, team 
membership, and academic performance. Further, I will conduct 
simultaneous hierarchical regression analysis to determine the 
unique contribution of each construct to predict exam scores. For 
achievement goals, motivational constructs, and engagement 
scales, I plan to use multivariate repeated measures ANOVA to 
determine the changes in students’ achievement goals, 
motivational constructs, and engagement scale after creating a 
learning environment combined with mobile technologies and 
instructional strategies in a first-year engineering class. As the 
engagement scale is not validated for undergraduate students but 
is extensively used in math and science classrooms, I plan to 
validate the instrument before progressing with the data analysis. 


7. IMPLICATIONS OF THE STUDY 


I will discuss the findings of these results with particular 
emphasis on challenges associated with such a learning 
environment. Further, I will discuss the study findings to indicate 
both the positive and negative effects of such integration of 
educational technologies and instructional strategies. Further, the 
study implications will advance the literature of computer science 
and engineering education by suggesting how educational 
technologies can be the way to incorporate instructional 
strategies in the classroom. Also, how this integration can help to 
set up a constructive social environment which may improve 
students learning and performance, and enhance self-regulated 
learning skills. 


ACKNOWLEDGMENTS 


This work is supported by the grant from the Institute of 
Education Sciences (IES) award number R305A180477. I 
acknowledge the support of my advisor and committee members. 


REFERENCES 


{1] Xiangmin Fan., Wencan Luo., Muhsin Menekse, Diane Litman, and Jingtao 
Wang, 2017. Scaling reflection prompts in large classrooms via mobile interfaces 
and natural language processing. In Proceedings of 22nd ACM Conference on 
Intelligent User Interfaces (UI 2017), Limassol, Cyprus. 

Xiangmin Fan., Wencan Luo., Muhsin Menekse, Diane Litman, and Jingtao 
Wang, 2015. CourseMIRROR: Enhancing large classroom instructor-student 
interactions via mobile interfaces and natural language processing. Works-In- 
Progress, In Proceedings of ACM Conference on Human Factors in Computing 
Systems (CHI 2015), 1473-1478, Seoul, Korea. 

Misty L. Loughry, Matthew W Ohland, and David J Woehr, 2014. Assessing 
teamwork skills for assurance of learning using CATME team tools. Journal of 
Marketing Education 36 (1): 5-19. 

Matthew W Ohland, Misty L. Loughry, David J. Woehr, Lisa G. Bullard, Richard 
M. Felder, Cynthia J. Finelli, Richard A. Layton, Hal R. Pomeranz, Douglas G. 
Schmucker, 2005. The comprehensive assessment of team-member 
effectiveness.” Software Created as Part of NSF DUE-ASA Award 243254. 


[2] 


